1 Data preparation

1.1 Outline

  • Load scripts: loads libraries and useful scripts used in the analyses; all .R files contained in scripts at the root of the factory are automatically loaded

  • Load data: imports datasets, and may contain some ad hoc changes to the data such as specific data cleaning (not used in other reports), new variables used in the analyses, etc.

1.2 Load packages


library(reportfactory)
library(here)
library(rio) 
library(tidyverse)
library(incidence)
library(distcrete)
library(epitrix)
library(earlyR)
library(projections)
library(linelist)
library(remotes)
library(janitor)
library(kableExtra)
library(DT)
library(cyphr)
library(chngpt)
library(lubridate)
library(ggpubr)
library(ggnewscale)

1.3 Load scripts

These scripts will load:

  • all scripts stored as .R files inside /scripts/
  • all scripts stored as .R files inside /src/

These scripts also contain routines to access the latest clean encrypted data (see next section).


reportfactory::rfh_load_scripts()

1.4 Load clean data

We import the latest NHS pathways data:


x <- import_pathways() %>%
  as_tibble()
x
## # A tibble: 183,045 x 11
##    site_type date       sex   age   ccg_code ccg_name count postcode nhs_region
##    <chr>     <date>     <chr> <chr> <chr>    <chr>    <int> <chr>    <chr>     
##  1 111       2020-03-18 fema… miss… e380000… nhs_glo…     1 gl34fe   South West
##  2 111       2020-03-18 fema… miss… e380001… nhs_sou…     1 ne325nn  North Eas…
##  3 111       2020-03-18 fema… 0-18  e380000… nhs_air…     8 bd57jr   North Eas…
##  4 111       2020-03-18 fema… 0-18  e380000… nhs_ash…     7 tn254ab  South East
##  5 111       2020-03-18 fema… 0-18  e380000… nhs_bar…    35 rm13ae   London    
##  6 111       2020-03-18 fema… 0-18  e380000… nhs_bar…     9 n111np   London    
##  7 111       2020-03-18 fema… 0-18  e380000… nhs_bar…    11 s752py   North Eas…
##  8 111       2020-03-18 fema… 0-18  e380000… nhs_bas…    19 ss143hg  East of E…
##  9 111       2020-03-18 fema… 0-18  e380000… nhs_bas…     6 dn227xf  North Eas…
## 10 111       2020-03-18 fema… 0-18  e380000… nhs_bat…     9 ba25rp   South West
## # … with 183,035 more rows, and 2 more variables: day <int>, weekday <fct>

We also import demographics data for NHS regions in England, used later in our analysis:


path <- here::here("data", "csv", "nhs_region_population_2018.csv")
nhs_region_pop <- rio::import(path) %>%
  mutate(nhs_region = str_to_title(gsub("_"," ",nhs_region)))

nhs_region_pop$nhs_region <- gsub(" Of ", " of ", nhs_region_pop$nhs_region)
nhs_region_pop$nhs_region <- gsub(" And ", " and ", nhs_region_pop$nhs_region)
nhs_region_pop
##                  nhs_region variable      value
## 1                North West     0-18 0.22538599
## 2  North East and Yorkshire     0-18 0.21876449
## 3                  Midlands     0-18 0.22564656
## 4           East of England     0-18 0.22810783
## 5                    London     0-18 0.23764782
## 6                South East     0-18 0.22458811
## 7                South West     0-18 0.20799797
## 8                North West    19-69 0.64274078
## 9  North East and Yorkshire    19-69 0.64437753
## 10                 Midlands    19-69 0.63876675
## 11          East of England    19-69 0.63034229
## 12                   London    19-69 0.67820084
## 13               South East    19-69 0.63267336
## 14               South West    19-69 0.63176131
## 15               North West   70-120 0.13187323
## 16 North East and Yorkshire   70-120 0.13685797
## 17                 Midlands   70-120 0.13558669
## 18          East of England   70-120 0.14154988
## 19                   London   70-120 0.08415135
## 20               South East   70-120 0.14273853
## 21               South West   70-120 0.16024072

Finally, we import publically available deaths per NHS region:


dth <- import_deaths() %>%
  mutate(nhs_region = str_to_title(gsub("_"," ",nhs_region)))

#truncation to account for reporting delay
delay_max <- 21

dth$nhs_region <- gsub(" Of ", " of ", dth$nhs_region)
dth$nhs_region <- gsub(" And ", " and ", dth$nhs_region)
dth
##     date_report               nhs_region deaths
## 1    2020-03-01          East of England      0
## 2    2020-03-02          East of England      1
## 3    2020-03-03          East of England      0
## 4    2020-03-04          East of England      0
## 5    2020-03-05          East of England      0
## 6    2020-03-06          East of England      1
## 7    2020-03-07          East of England      0
## 8    2020-03-08          East of England      0
## 9    2020-03-09          East of England      1
## 10   2020-03-10          East of England      0
## 11   2020-03-11          East of England      0
## 12   2020-03-12          East of England      0
## 13   2020-03-13          East of England      1
## 14   2020-03-14          East of England      2
## 15   2020-03-15          East of England      2
## 16   2020-03-16          East of England      1
## 17   2020-03-17          East of England      1
## 18   2020-03-18          East of England      5
## 19   2020-03-19          East of England      4
## 20   2020-03-20          East of England      2
## 21   2020-03-21          East of England     11
## 22   2020-03-22          East of England     12
## 23   2020-03-23          East of England     11
## 24   2020-03-24          East of England     19
## 25   2020-03-25          East of England     26
## 26   2020-03-26          East of England     36
## 27   2020-03-27          East of England     38
## 28   2020-03-28          East of England     28
## 29   2020-03-29          East of England     43
## 30   2020-03-30          East of England     45
## 31   2020-03-31          East of England     70
## 32   2020-04-01          East of England     62
## 33   2020-04-02          East of England     65
## 34   2020-04-03          East of England     80
## 35   2020-04-04          East of England     71
## 36   2020-04-05          East of England     76
## 37   2020-04-06          East of England     71
## 38   2020-04-07          East of England     93
## 39   2020-04-08          East of England    111
## 40   2020-04-09          East of England     87
## 41   2020-04-10          East of England     74
## 42   2020-04-11          East of England     92
## 43   2020-04-12          East of England    100
## 44   2020-04-13          East of England     78
## 45   2020-04-14          East of England     61
## 46   2020-04-15          East of England     82
## 47   2020-04-16          East of England     74
## 48   2020-04-17          East of England     86
## 49   2020-04-18          East of England     64
## 50   2020-04-19          East of England     67
## 51   2020-04-20          East of England     67
## 52   2020-04-21          East of England     75
## 53   2020-04-22          East of England     67
## 54   2020-04-23          East of England     49
## 55   2020-04-24          East of England     66
## 56   2020-04-25          East of England     54
## 57   2020-04-26          East of England     48
## 58   2020-04-27          East of England     46
## 59   2020-04-28          East of England     58
## 60   2020-04-29          East of England     32
## 61   2020-04-30          East of England     45
## 62   2020-05-01          East of England     49
## 63   2020-05-02          East of England     29
## 64   2020-05-03          East of England     41
## 65   2020-05-04          East of England     19
## 66   2020-05-05          East of England     36
## 67   2020-05-06          East of England     31
## 68   2020-05-07          East of England     33
## 69   2020-05-08          East of England     33
## 70   2020-05-09          East of England     29
## 71   2020-05-10          East of England     22
## 72   2020-05-11          East of England     18
## 73   2020-05-12          East of England     21
## 74   2020-05-13          East of England     27
## 75   2020-05-14          East of England     26
## 76   2020-05-15          East of England     19
## 77   2020-05-16          East of England     26
## 78   2020-05-17          East of England     17
## 79   2020-05-18          East of England     25
## 80   2020-05-19          East of England     15
## 81   2020-05-20          East of England     26
## 82   2020-05-21          East of England     21
## 83   2020-05-22          East of England     13
## 84   2020-05-23          East of England     12
## 85   2020-05-24          East of England     17
## 86   2020-05-25          East of England     25
## 87   2020-05-26          East of England     14
## 88   2020-05-27          East of England     12
## 89   2020-05-28          East of England     17
## 90   2020-05-29          East of England     16
## 91   2020-05-30          East of England      9
## 92   2020-05-31          East of England      8
## 93   2020-06-01          East of England     17
## 94   2020-06-02          East of England     14
## 95   2020-06-03          East of England     10
## 96   2020-06-04          East of England      7
## 97   2020-06-05          East of England     14
## 98   2020-06-06          East of England      5
## 99   2020-06-07          East of England      9
## 100  2020-06-08          East of England      7
## 101  2020-06-09          East of England      6
## 102  2020-06-10          East of England      8
## 103  2020-06-11          East of England      1
## 104  2020-06-12          East of England      9
## 105  2020-06-13          East of England      5
## 106  2020-06-14          East of England      4
## 107  2020-06-15          East of England      8
## 108  2020-06-16          East of England      3
## 109  2020-06-17          East of England      7
## 110  2020-06-18          East of England      4
## 111  2020-06-19          East of England      7
## 112  2020-06-20          East of England      4
## 113  2020-06-21          East of England      3
## 114  2020-06-22          East of England      6
## 115  2020-06-23          East of England      5
## 116  2020-06-24          East of England      4
## 117  2020-06-25          East of England      1
## 118  2020-06-26          East of England      5
## 119  2020-06-27          East of England      6
## 120  2020-06-28          East of England      8
## 121  2020-06-29          East of England      4
## 122  2020-06-30          East of England      5
## 123  2020-07-01          East of England      2
## 124  2020-07-02          East of England      5
## 125  2020-07-03          East of England      0
## 126  2020-07-04          East of England      3
## 127  2020-07-05          East of England      1
## 128  2020-07-06          East of England      2
## 129  2020-07-07          East of England      2
## 130  2020-07-08          East of England      0
## 131  2020-07-09          East of England      8
## 132  2020-07-10          East of England      4
## 133  2020-07-11          East of England      2
## 134  2020-07-12          East of England      1
## 135  2020-07-13          East of England      7
## 136  2020-07-14          East of England      2
## 137  2020-07-15          East of England      0
## 138  2020-07-16          East of England      0
## 139  2020-07-17          East of England      0
## 140  2020-07-18          East of England      0
## 141  2020-07-19          East of England      0
## 142  2020-03-01                   London      0
## 143  2020-03-02                   London      0
## 144  2020-03-03                   London      0
## 145  2020-03-04                   London      0
## 146  2020-03-05                   London      0
## 147  2020-03-06                   London      1
## 148  2020-03-07                   London      0
## 149  2020-03-08                   London      0
## 150  2020-03-09                   London      1
## 151  2020-03-10                   London      0
## 152  2020-03-11                   London      5
## 153  2020-03-12                   London      6
## 154  2020-03-13                   London     10
## 155  2020-03-14                   London     13
## 156  2020-03-15                   London      9
## 157  2020-03-16                   London     15
## 158  2020-03-17                   London     23
## 159  2020-03-18                   London     27
## 160  2020-03-19                   London     25
## 161  2020-03-20                   London     44
## 162  2020-03-21                   London     49
## 163  2020-03-22                   London     54
## 164  2020-03-23                   London     63
## 165  2020-03-24                   London     86
## 166  2020-03-25                   London    112
## 167  2020-03-26                   London    129
## 168  2020-03-27                   London    129
## 169  2020-03-28                   London    122
## 170  2020-03-29                   London    145
## 171  2020-03-30                   London    149
## 172  2020-03-31                   London    181
## 173  2020-04-01                   London    202
## 174  2020-04-02                   London    191
## 175  2020-04-03                   London    196
## 176  2020-04-04                   London    230
## 177  2020-04-05                   London    195
## 178  2020-04-06                   London    197
## 179  2020-04-07                   London    220
## 180  2020-04-08                   London    238
## 181  2020-04-09                   London    206
## 182  2020-04-10                   London    170
## 183  2020-04-11                   London    178
## 184  2020-04-12                   London    158
## 185  2020-04-13                   London    166
## 186  2020-04-14                   London    143
## 187  2020-04-15                   London    142
## 188  2020-04-16                   London    140
## 189  2020-04-17                   London    100
## 190  2020-04-18                   London    101
## 191  2020-04-19                   London    103
## 192  2020-04-20                   London     95
## 193  2020-04-21                   London     94
## 194  2020-04-22                   London    109
## 195  2020-04-23                   London     77
## 196  2020-04-24                   London     71
## 197  2020-04-25                   London     58
## 198  2020-04-26                   London     53
## 199  2020-04-27                   London     51
## 200  2020-04-28                   London     44
## 201  2020-04-29                   London     45
## 202  2020-04-30                   London     40
## 203  2020-05-01                   London     41
## 204  2020-05-02                   London     41
## 205  2020-05-03                   London     36
## 206  2020-05-04                   London     30
## 207  2020-05-05                   London     25
## 208  2020-05-06                   London     37
## 209  2020-05-07                   London     37
## 210  2020-05-08                   London     30
## 211  2020-05-09                   London     23
## 212  2020-05-10                   London     26
## 213  2020-05-11                   London     18
## 214  2020-05-12                   London     18
## 215  2020-05-13                   London     17
## 216  2020-05-14                   London     20
## 217  2020-05-15                   London     18
## 218  2020-05-16                   London     14
## 219  2020-05-17                   London     15
## 220  2020-05-18                   London     10
## 221  2020-05-19                   London     14
## 222  2020-05-20                   London     19
## 223  2020-05-21                   London     12
## 224  2020-05-22                   London     10
## 225  2020-05-23                   London      6
## 226  2020-05-24                   London      7
## 227  2020-05-25                   London      9
## 228  2020-05-26                   London     13
## 229  2020-05-27                   London      7
## 230  2020-05-28                   London      8
## 231  2020-05-29                   London      7
## 232  2020-05-30                   London     12
## 233  2020-05-31                   London      6
## 234  2020-06-01                   London     10
## 235  2020-06-02                   London      8
## 236  2020-06-03                   London      6
## 237  2020-06-04                   London      8
## 238  2020-06-05                   London      4
## 239  2020-06-06                   London      0
## 240  2020-06-07                   London      5
## 241  2020-06-08                   London      5
## 242  2020-06-09                   London      4
## 243  2020-06-10                   London      7
## 244  2020-06-11                   London      5
## 245  2020-06-12                   London      3
## 246  2020-06-13                   London      3
## 247  2020-06-14                   London      3
## 248  2020-06-15                   London      1
## 249  2020-06-16                   London      2
## 250  2020-06-17                   London      1
## 251  2020-06-18                   London      2
## 252  2020-06-19                   London      5
## 253  2020-06-20                   London      3
## 254  2020-06-21                   London      4
## 255  2020-06-22                   London      2
## 256  2020-06-23                   London      1
## 257  2020-06-24                   London      4
## 258  2020-06-25                   London      3
## 259  2020-06-26                   London      2
## 260  2020-06-27                   London      1
## 261  2020-06-28                   London      2
## 262  2020-06-29                   London      2
## 263  2020-06-30                   London      1
## 264  2020-07-01                   London      2
## 265  2020-07-02                   London      2
## 266  2020-07-03                   London      2
## 267  2020-07-04                   London      1
## 268  2020-07-05                   London      3
## 269  2020-07-06                   London      2
## 270  2020-07-07                   London      1
## 271  2020-07-08                   London      3
## 272  2020-07-09                   London      4
## 273  2020-07-10                   London      0
## 274  2020-07-11                   London      0
## 275  2020-07-12                   London      0
## 276  2020-07-13                   London      1
## 277  2020-07-14                   London      0
## 278  2020-07-15                   London      1
## 279  2020-07-16                   London      0
## 280  2020-07-17                   London      0
## 281  2020-07-18                   London      0
## 282  2020-07-19                   London      0
## 283  2020-03-01                 Midlands      0
## 284  2020-03-02                 Midlands      0
## 285  2020-03-03                 Midlands      1
## 286  2020-03-04                 Midlands      0
## 287  2020-03-05                 Midlands      0
## 288  2020-03-06                 Midlands      0
## 289  2020-03-07                 Midlands      0
## 290  2020-03-08                 Midlands      2
## 291  2020-03-09                 Midlands      1
## 292  2020-03-10                 Midlands      0
## 293  2020-03-11                 Midlands      2
## 294  2020-03-12                 Midlands      6
## 295  2020-03-13                 Midlands      5
## 296  2020-03-14                 Midlands      4
## 297  2020-03-15                 Midlands      5
## 298  2020-03-16                 Midlands     11
## 299  2020-03-17                 Midlands      8
## 300  2020-03-18                 Midlands     13
## 301  2020-03-19                 Midlands      8
## 302  2020-03-20                 Midlands     28
## 303  2020-03-21                 Midlands     13
## 304  2020-03-22                 Midlands     31
## 305  2020-03-23                 Midlands     33
## 306  2020-03-24                 Midlands     41
## 307  2020-03-25                 Midlands     48
## 308  2020-03-26                 Midlands     64
## 309  2020-03-27                 Midlands     72
## 310  2020-03-28                 Midlands     89
## 311  2020-03-29                 Midlands     92
## 312  2020-03-30                 Midlands     90
## 313  2020-03-31                 Midlands    123
## 314  2020-04-01                 Midlands    140
## 315  2020-04-02                 Midlands    142
## 316  2020-04-03                 Midlands    124
## 317  2020-04-04                 Midlands    151
## 318  2020-04-05                 Midlands    164
## 319  2020-04-06                 Midlands    140
## 320  2020-04-07                 Midlands    123
## 321  2020-04-08                 Midlands    186
## 322  2020-04-09                 Midlands    139
## 323  2020-04-10                 Midlands    127
## 324  2020-04-11                 Midlands    142
## 325  2020-04-12                 Midlands    139
## 326  2020-04-13                 Midlands    120
## 327  2020-04-14                 Midlands    116
## 328  2020-04-15                 Midlands    147
## 329  2020-04-16                 Midlands    102
## 330  2020-04-17                 Midlands    118
## 331  2020-04-18                 Midlands    115
## 332  2020-04-19                 Midlands     92
## 333  2020-04-20                 Midlands    107
## 334  2020-04-21                 Midlands     86
## 335  2020-04-22                 Midlands     78
## 336  2020-04-23                 Midlands    103
## 337  2020-04-24                 Midlands     79
## 338  2020-04-25                 Midlands     72
## 339  2020-04-26                 Midlands     81
## 340  2020-04-27                 Midlands     74
## 341  2020-04-28                 Midlands     68
## 342  2020-04-29                 Midlands     53
## 343  2020-04-30                 Midlands     56
## 344  2020-05-01                 Midlands     64
## 345  2020-05-02                 Midlands     51
## 346  2020-05-03                 Midlands     52
## 347  2020-05-04                 Midlands     61
## 348  2020-05-05                 Midlands     59
## 349  2020-05-06                 Midlands     59
## 350  2020-05-07                 Midlands     48
## 351  2020-05-08                 Midlands     34
## 352  2020-05-09                 Midlands     37
## 353  2020-05-10                 Midlands     42
## 354  2020-05-11                 Midlands     33
## 355  2020-05-12                 Midlands     45
## 356  2020-05-13                 Midlands     40
## 357  2020-05-14                 Midlands     38
## 358  2020-05-15                 Midlands     40
## 359  2020-05-16                 Midlands     34
## 360  2020-05-17                 Midlands     31
## 361  2020-05-18                 Midlands     36
## 362  2020-05-19                 Midlands     35
## 363  2020-05-20                 Midlands     36
## 364  2020-05-21                 Midlands     32
## 365  2020-05-22                 Midlands     27
## 366  2020-05-23                 Midlands     34
## 367  2020-05-24                 Midlands     20
## 368  2020-05-25                 Midlands     26
## 369  2020-05-26                 Midlands     33
## 370  2020-05-27                 Midlands     29
## 371  2020-05-28                 Midlands     28
## 372  2020-05-29                 Midlands     20
## 373  2020-05-30                 Midlands     21
## 374  2020-05-31                 Midlands     22
## 375  2020-06-01                 Midlands     20
## 376  2020-06-02                 Midlands     22
## 377  2020-06-03                 Midlands     24
## 378  2020-06-04                 Midlands     16
## 379  2020-06-05                 Midlands     21
## 380  2020-06-06                 Midlands     20
## 381  2020-06-07                 Midlands     17
## 382  2020-06-08                 Midlands     16
## 383  2020-06-09                 Midlands     18
## 384  2020-06-10                 Midlands     15
## 385  2020-06-11                 Midlands     13
## 386  2020-06-12                 Midlands     12
## 387  2020-06-13                 Midlands      6
## 388  2020-06-14                 Midlands     18
## 389  2020-06-15                 Midlands     12
## 390  2020-06-16                 Midlands     15
## 391  2020-06-17                 Midlands     11
## 392  2020-06-18                 Midlands     15
## 393  2020-06-19                 Midlands     10
## 394  2020-06-20                 Midlands     15
## 395  2020-06-21                 Midlands     14
## 396  2020-06-22                 Midlands     14
## 397  2020-06-23                 Midlands     16
## 398  2020-06-24                 Midlands     15
## 399  2020-06-25                 Midlands     18
## 400  2020-06-26                 Midlands      5
## 401  2020-06-27                 Midlands      5
## 402  2020-06-28                 Midlands      7
## 403  2020-06-29                 Midlands      6
## 404  2020-06-30                 Midlands      6
## 405  2020-07-01                 Midlands      7
## 406  2020-07-02                 Midlands      9
## 407  2020-07-03                 Midlands      3
## 408  2020-07-04                 Midlands      4
## 409  2020-07-05                 Midlands      6
## 410  2020-07-06                 Midlands      5
## 411  2020-07-07                 Midlands      3
## 412  2020-07-08                 Midlands      5
## 413  2020-07-09                 Midlands      8
## 414  2020-07-10                 Midlands      3
## 415  2020-07-11                 Midlands      0
## 416  2020-07-12                 Midlands      5
## 417  2020-07-13                 Midlands      1
## 418  2020-07-14                 Midlands      1
## 419  2020-07-15                 Midlands      6
## 420  2020-07-16                 Midlands      1
## 421  2020-07-17                 Midlands      1
## 422  2020-07-18                 Midlands      1
## 423  2020-07-19                 Midlands      1
## 424  2020-03-01 North East and Yorkshire      0
## 425  2020-03-02 North East and Yorkshire      0
## 426  2020-03-03 North East and Yorkshire      0
## 427  2020-03-04 North East and Yorkshire      0
## 428  2020-03-05 North East and Yorkshire      0
## 429  2020-03-06 North East and Yorkshire      0
## 430  2020-03-07 North East and Yorkshire      0
## 431  2020-03-08 North East and Yorkshire      0
## 432  2020-03-09 North East and Yorkshire      0
## 433  2020-03-10 North East and Yorkshire      0
## 434  2020-03-11 North East and Yorkshire      0
## 435  2020-03-12 North East and Yorkshire      0
## 436  2020-03-13 North East and Yorkshire      0
## 437  2020-03-14 North East and Yorkshire      0
## 438  2020-03-15 North East and Yorkshire      2
## 439  2020-03-16 North East and Yorkshire      3
## 440  2020-03-17 North East and Yorkshire      1
## 441  2020-03-18 North East and Yorkshire      2
## 442  2020-03-19 North East and Yorkshire      6
## 443  2020-03-20 North East and Yorkshire      5
## 444  2020-03-21 North East and Yorkshire      6
## 445  2020-03-22 North East and Yorkshire      7
## 446  2020-03-23 North East and Yorkshire      9
## 447  2020-03-24 North East and Yorkshire      8
## 448  2020-03-25 North East and Yorkshire     18
## 449  2020-03-26 North East and Yorkshire     21
## 450  2020-03-27 North East and Yorkshire     28
## 451  2020-03-28 North East and Yorkshire     35
## 452  2020-03-29 North East and Yorkshire     38
## 453  2020-03-30 North East and Yorkshire     64
## 454  2020-03-31 North East and Yorkshire     60
## 455  2020-04-01 North East and Yorkshire     67
## 456  2020-04-02 North East and Yorkshire     75
## 457  2020-04-03 North East and Yorkshire    100
## 458  2020-04-04 North East and Yorkshire    105
## 459  2020-04-05 North East and Yorkshire     92
## 460  2020-04-06 North East and Yorkshire     96
## 461  2020-04-07 North East and Yorkshire    102
## 462  2020-04-08 North East and Yorkshire    107
## 463  2020-04-09 North East and Yorkshire    111
## 464  2020-04-10 North East and Yorkshire    117
## 465  2020-04-11 North East and Yorkshire     98
## 466  2020-04-12 North East and Yorkshire     84
## 467  2020-04-13 North East and Yorkshire     94
## 468  2020-04-14 North East and Yorkshire    107
## 469  2020-04-15 North East and Yorkshire     96
## 470  2020-04-16 North East and Yorkshire    103
## 471  2020-04-17 North East and Yorkshire     88
## 472  2020-04-18 North East and Yorkshire     95
## 473  2020-04-19 North East and Yorkshire     88
## 474  2020-04-20 North East and Yorkshire    100
## 475  2020-04-21 North East and Yorkshire     76
## 476  2020-04-22 North East and Yorkshire     84
## 477  2020-04-23 North East and Yorkshire     63
## 478  2020-04-24 North East and Yorkshire     72
## 479  2020-04-25 North East and Yorkshire     69
## 480  2020-04-26 North East and Yorkshire     65
## 481  2020-04-27 North East and Yorkshire     65
## 482  2020-04-28 North East and Yorkshire     57
## 483  2020-04-29 North East and Yorkshire     69
## 484  2020-04-30 North East and Yorkshire     57
## 485  2020-05-01 North East and Yorkshire     64
## 486  2020-05-02 North East and Yorkshire     48
## 487  2020-05-03 North East and Yorkshire     40
## 488  2020-05-04 North East and Yorkshire     49
## 489  2020-05-05 North East and Yorkshire     40
## 490  2020-05-06 North East and Yorkshire     51
## 491  2020-05-07 North East and Yorkshire     45
## 492  2020-05-08 North East and Yorkshire     42
## 493  2020-05-09 North East and Yorkshire     44
## 494  2020-05-10 North East and Yorkshire     40
## 495  2020-05-11 North East and Yorkshire     29
## 496  2020-05-12 North East and Yorkshire     27
## 497  2020-05-13 North East and Yorkshire     28
## 498  2020-05-14 North East and Yorkshire     31
## 499  2020-05-15 North East and Yorkshire     32
## 500  2020-05-16 North East and Yorkshire     35
## 501  2020-05-17 North East and Yorkshire     26
## 502  2020-05-18 North East and Yorkshire     30
## 503  2020-05-19 North East and Yorkshire     27
## 504  2020-05-20 North East and Yorkshire     22
## 505  2020-05-21 North East and Yorkshire     33
## 506  2020-05-22 North East and Yorkshire     22
## 507  2020-05-23 North East and Yorkshire     18
## 508  2020-05-24 North East and Yorkshire     26
## 509  2020-05-25 North East and Yorkshire     21
## 510  2020-05-26 North East and Yorkshire     21
## 511  2020-05-27 North East and Yorkshire     22
## 512  2020-05-28 North East and Yorkshire     21
## 513  2020-05-29 North East and Yorkshire     25
## 514  2020-05-30 North East and Yorkshire     20
## 515  2020-05-31 North East and Yorkshire     20
## 516  2020-06-01 North East and Yorkshire     17
## 517  2020-06-02 North East and Yorkshire     23
## 518  2020-06-03 North East and Yorkshire     23
## 519  2020-06-04 North East and Yorkshire     17
## 520  2020-06-05 North East and Yorkshire     18
## 521  2020-06-06 North East and Yorkshire     21
## 522  2020-06-07 North East and Yorkshire     14
## 523  2020-06-08 North East and Yorkshire     11
## 524  2020-06-09 North East and Yorkshire     12
## 525  2020-06-10 North East and Yorkshire     19
## 526  2020-06-11 North East and Yorkshire      7
## 527  2020-06-12 North East and Yorkshire      9
## 528  2020-06-13 North East and Yorkshire     10
## 529  2020-06-14 North East and Yorkshire     11
## 530  2020-06-15 North East and Yorkshire      9
## 531  2020-06-16 North East and Yorkshire     10
## 532  2020-06-17 North East and Yorkshire      9
## 533  2020-06-18 North East and Yorkshire     11
## 534  2020-06-19 North East and Yorkshire      6
## 535  2020-06-20 North East and Yorkshire      5
## 536  2020-06-21 North East and Yorkshire      4
## 537  2020-06-22 North East and Yorkshire      6
## 538  2020-06-23 North East and Yorkshire      7
## 539  2020-06-24 North East and Yorkshire     10
## 540  2020-06-25 North East and Yorkshire      4
## 541  2020-06-26 North East and Yorkshire      7
## 542  2020-06-27 North East and Yorkshire      3
## 543  2020-06-28 North East and Yorkshire      5
## 544  2020-06-29 North East and Yorkshire      2
## 545  2020-06-30 North East and Yorkshire      5
## 546  2020-07-01 North East and Yorkshire      1
## 547  2020-07-02 North East and Yorkshire      4
## 548  2020-07-03 North East and Yorkshire      3
## 549  2020-07-04 North East and Yorkshire      4
## 550  2020-07-05 North East and Yorkshire      2
## 551  2020-07-06 North East and Yorkshire      2
## 552  2020-07-07 North East and Yorkshire      3
## 553  2020-07-08 North East and Yorkshire      3
## 554  2020-07-09 North East and Yorkshire      0
## 555  2020-07-10 North East and Yorkshire      3
## 556  2020-07-11 North East and Yorkshire      1
## 557  2020-07-12 North East and Yorkshire      4
## 558  2020-07-13 North East and Yorkshire      1
## 559  2020-07-14 North East and Yorkshire      1
## 560  2020-07-15 North East and Yorkshire      2
## 561  2020-07-16 North East and Yorkshire      2
## 562  2020-07-17 North East and Yorkshire      1
## 563  2020-07-18 North East and Yorkshire      2
## 564  2020-07-19 North East and Yorkshire      0
## 565  2020-03-01               North West      0
## 566  2020-03-02               North West      0
## 567  2020-03-03               North West      0
## 568  2020-03-04               North West      0
## 569  2020-03-05               North West      1
## 570  2020-03-06               North West      0
## 571  2020-03-07               North West      0
## 572  2020-03-08               North West      1
## 573  2020-03-09               North West      0
## 574  2020-03-10               North West      0
## 575  2020-03-11               North West      0
## 576  2020-03-12               North West      2
## 577  2020-03-13               North West      3
## 578  2020-03-14               North West      1
## 579  2020-03-15               North West      4
## 580  2020-03-16               North West      2
## 581  2020-03-17               North West      4
## 582  2020-03-18               North West      6
## 583  2020-03-19               North West      7
## 584  2020-03-20               North West     10
## 585  2020-03-21               North West     11
## 586  2020-03-22               North West     13
## 587  2020-03-23               North West     15
## 588  2020-03-24               North West     21
## 589  2020-03-25               North West     21
## 590  2020-03-26               North West     29
## 591  2020-03-27               North West     36
## 592  2020-03-28               North West     28
## 593  2020-03-29               North West     46
## 594  2020-03-30               North West     67
## 595  2020-03-31               North West     52
## 596  2020-04-01               North West     86
## 597  2020-04-02               North West     96
## 598  2020-04-03               North West     95
## 599  2020-04-04               North West     98
## 600  2020-04-05               North West    102
## 601  2020-04-06               North West    100
## 602  2020-04-07               North West    135
## 603  2020-04-08               North West    127
## 604  2020-04-09               North West    119
## 605  2020-04-10               North West    117
## 606  2020-04-11               North West    138
## 607  2020-04-12               North West    125
## 608  2020-04-13               North West    129
## 609  2020-04-14               North West    131
## 610  2020-04-15               North West    114
## 611  2020-04-16               North West    135
## 612  2020-04-17               North West     98
## 613  2020-04-18               North West    113
## 614  2020-04-19               North West     71
## 615  2020-04-20               North West     83
## 616  2020-04-21               North West     76
## 617  2020-04-22               North West     86
## 618  2020-04-23               North West     85
## 619  2020-04-24               North West     66
## 620  2020-04-25               North West     66
## 621  2020-04-26               North West     55
## 622  2020-04-27               North West     54
## 623  2020-04-28               North West     57
## 624  2020-04-29               North West     63
## 625  2020-04-30               North West     59
## 626  2020-05-01               North West     45
## 627  2020-05-02               North West     56
## 628  2020-05-03               North West     55
## 629  2020-05-04               North West     48
## 630  2020-05-05               North West     48
## 631  2020-05-06               North West     44
## 632  2020-05-07               North West     49
## 633  2020-05-08               North West     42
## 634  2020-05-09               North West     31
## 635  2020-05-10               North West     42
## 636  2020-05-11               North West     35
## 637  2020-05-12               North West     38
## 638  2020-05-13               North West     25
## 639  2020-05-14               North West     26
## 640  2020-05-15               North West     33
## 641  2020-05-16               North West     32
## 642  2020-05-17               North West     24
## 643  2020-05-18               North West     31
## 644  2020-05-19               North West     35
## 645  2020-05-20               North West     27
## 646  2020-05-21               North West     27
## 647  2020-05-22               North West     26
## 648  2020-05-23               North West     31
## 649  2020-05-24               North West     26
## 650  2020-05-25               North West     31
## 651  2020-05-26               North West     27
## 652  2020-05-27               North West     27
## 653  2020-05-28               North West     28
## 654  2020-05-29               North West     20
## 655  2020-05-30               North West     19
## 656  2020-05-31               North West     13
## 657  2020-06-01               North West     12
## 658  2020-06-02               North West     27
## 659  2020-06-03               North West     22
## 660  2020-06-04               North West     22
## 661  2020-06-05               North West     16
## 662  2020-06-06               North West     26
## 663  2020-06-07               North West     20
## 664  2020-06-08               North West     23
## 665  2020-06-09               North West     17
## 666  2020-06-10               North West     16
## 667  2020-06-11               North West     16
## 668  2020-06-12               North West     11
## 669  2020-06-13               North West     10
## 670  2020-06-14               North West     15
## 671  2020-06-15               North West     16
## 672  2020-06-16               North West     15
## 673  2020-06-17               North West     13
## 674  2020-06-18               North West     13
## 675  2020-06-19               North West      7
## 676  2020-06-20               North West     11
## 677  2020-06-21               North West      7
## 678  2020-06-22               North West     11
## 679  2020-06-23               North West     13
## 680  2020-06-24               North West     13
## 681  2020-06-25               North West     15
## 682  2020-06-26               North West      6
## 683  2020-06-27               North West      7
## 684  2020-06-28               North West      9
## 685  2020-06-29               North West      8
## 686  2020-06-30               North West      6
## 687  2020-07-01               North West      3
## 688  2020-07-02               North West      6
## 689  2020-07-03               North West      6
## 690  2020-07-04               North West      4
## 691  2020-07-05               North West      6
## 692  2020-07-06               North West      9
## 693  2020-07-07               North West      7
## 694  2020-07-08               North West      5
## 695  2020-07-09               North West     10
## 696  2020-07-10               North West      2
## 697  2020-07-11               North West      3
## 698  2020-07-12               North West      0
## 699  2020-07-13               North West      6
## 700  2020-07-14               North West      2
## 701  2020-07-15               North West      4
## 702  2020-07-16               North West      2
## 703  2020-07-17               North West      1
## 704  2020-07-18               North West      2
## 705  2020-07-19               North West      0
## 706  2020-03-01               South East      0
## 707  2020-03-02               South East      0
## 708  2020-03-03               South East      1
## 709  2020-03-04               South East      0
## 710  2020-03-05               South East      1
## 711  2020-03-06               South East      0
## 712  2020-03-07               South East      0
## 713  2020-03-08               South East      1
## 714  2020-03-09               South East      1
## 715  2020-03-10               South East      1
## 716  2020-03-11               South East      1
## 717  2020-03-12               South East      0
## 718  2020-03-13               South East      1
## 719  2020-03-14               South East      1
## 720  2020-03-15               South East      5
## 721  2020-03-16               South East      8
## 722  2020-03-17               South East      7
## 723  2020-03-18               South East     10
## 724  2020-03-19               South East      9
## 725  2020-03-20               South East     13
## 726  2020-03-21               South East      7
## 727  2020-03-22               South East     25
## 728  2020-03-23               South East     20
## 729  2020-03-24               South East     22
## 730  2020-03-25               South East     29
## 731  2020-03-26               South East     35
## 732  2020-03-27               South East     34
## 733  2020-03-28               South East     36
## 734  2020-03-29               South East     55
## 735  2020-03-30               South East     58
## 736  2020-03-31               South East     65
## 737  2020-04-01               South East     66
## 738  2020-04-02               South East     55
## 739  2020-04-03               South East     72
## 740  2020-04-04               South East     80
## 741  2020-04-05               South East     82
## 742  2020-04-06               South East     88
## 743  2020-04-07               South East    100
## 744  2020-04-08               South East     83
## 745  2020-04-09               South East    104
## 746  2020-04-10               South East     88
## 747  2020-04-11               South East     88
## 748  2020-04-12               South East     88
## 749  2020-04-13               South East     84
## 750  2020-04-14               South East     65
## 751  2020-04-15               South East     72
## 752  2020-04-16               South East     56
## 753  2020-04-17               South East     86
## 754  2020-04-18               South East     57
## 755  2020-04-19               South East     70
## 756  2020-04-20               South East     87
## 757  2020-04-21               South East     51
## 758  2020-04-22               South East     54
## 759  2020-04-23               South East     57
## 760  2020-04-24               South East     64
## 761  2020-04-25               South East     51
## 762  2020-04-26               South East     51
## 763  2020-04-27               South East     41
## 764  2020-04-28               South East     40
## 765  2020-04-29               South East     47
## 766  2020-04-30               South East     29
## 767  2020-05-01               South East     37
## 768  2020-05-02               South East     36
## 769  2020-05-03               South East     17
## 770  2020-05-04               South East     35
## 771  2020-05-05               South East     29
## 772  2020-05-06               South East     25
## 773  2020-05-07               South East     27
## 774  2020-05-08               South East     26
## 775  2020-05-09               South East     28
## 776  2020-05-10               South East     19
## 777  2020-05-11               South East     25
## 778  2020-05-12               South East     27
## 779  2020-05-13               South East     18
## 780  2020-05-14               South East     32
## 781  2020-05-15               South East     25
## 782  2020-05-16               South East     22
## 783  2020-05-17               South East     18
## 784  2020-05-18               South East     22
## 785  2020-05-19               South East     12
## 786  2020-05-20               South East     22
## 787  2020-05-21               South East     15
## 788  2020-05-22               South East     17
## 789  2020-05-23               South East     21
## 790  2020-05-24               South East     17
## 791  2020-05-25               South East     13
## 792  2020-05-26               South East     19
## 793  2020-05-27               South East     18
## 794  2020-05-28               South East     12
## 795  2020-05-29               South East     21
## 796  2020-05-30               South East      8
## 797  2020-05-31               South East     12
## 798  2020-06-01               South East     11
## 799  2020-06-02               South East     13
## 800  2020-06-03               South East     18
## 801  2020-06-04               South East     11
## 802  2020-06-05               South East     11
## 803  2020-06-06               South East     10
## 804  2020-06-07               South East     12
## 805  2020-06-08               South East      8
## 806  2020-06-09               South East     10
## 807  2020-06-10               South East     11
## 808  2020-06-11               South East      5
## 809  2020-06-12               South East      6
## 810  2020-06-13               South East      7
## 811  2020-06-14               South East      7
## 812  2020-06-15               South East      8
## 813  2020-06-16               South East     13
## 814  2020-06-17               South East      9
## 815  2020-06-18               South East      4
## 816  2020-06-19               South East      7
## 817  2020-06-20               South East      5
## 818  2020-06-21               South East      3
## 819  2020-06-22               South East      2
## 820  2020-06-23               South East      8
## 821  2020-06-24               South East      7
## 822  2020-06-25               South East      5
## 823  2020-06-26               South East      8
## 824  2020-06-27               South East      8
## 825  2020-06-28               South East      6
## 826  2020-06-29               South East      5
## 827  2020-06-30               South East      5
## 828  2020-07-01               South East      2
## 829  2020-07-02               South East      7
## 830  2020-07-03               South East      3
## 831  2020-07-04               South East      5
## 832  2020-07-05               South East      4
## 833  2020-07-06               South East      3
## 834  2020-07-07               South East      5
## 835  2020-07-08               South East      3
## 836  2020-07-09               South East      7
## 837  2020-07-10               South East      3
## 838  2020-07-11               South East      1
## 839  2020-07-12               South East      3
## 840  2020-07-13               South East      4
## 841  2020-07-14               South East      3
## 842  2020-07-15               South East      3
## 843  2020-07-16               South East      3
## 844  2020-07-17               South East      1
## 845  2020-07-18               South East      4
## 846  2020-07-19               South East      0
## 847  2020-03-01               South West      0
## 848  2020-03-02               South West      0
## 849  2020-03-03               South West      0
## 850  2020-03-04               South West      0
## 851  2020-03-05               South West      0
## 852  2020-03-06               South West      0
## 853  2020-03-07               South West      0
## 854  2020-03-08               South West      0
## 855  2020-03-09               South West      0
## 856  2020-03-10               South West      0
## 857  2020-03-11               South West      1
## 858  2020-03-12               South West      0
## 859  2020-03-13               South West      0
## 860  2020-03-14               South West      1
## 861  2020-03-15               South West      0
## 862  2020-03-16               South West      0
## 863  2020-03-17               South West      2
## 864  2020-03-18               South West      2
## 865  2020-03-19               South West      4
## 866  2020-03-20               South West      3
## 867  2020-03-21               South West      6
## 868  2020-03-22               South West      7
## 869  2020-03-23               South West      8
## 870  2020-03-24               South West      7
## 871  2020-03-25               South West      9
## 872  2020-03-26               South West     11
## 873  2020-03-27               South West     13
## 874  2020-03-28               South West     21
## 875  2020-03-29               South West     18
## 876  2020-03-30               South West     23
## 877  2020-03-31               South West     23
## 878  2020-04-01               South West     21
## 879  2020-04-02               South West     23
## 880  2020-04-03               South West     30
## 881  2020-04-04               South West     42
## 882  2020-04-05               South West     32
## 883  2020-04-06               South West     34
## 884  2020-04-07               South West     39
## 885  2020-04-08               South West     47
## 886  2020-04-09               South West     24
## 887  2020-04-10               South West     46
## 888  2020-04-11               South West     43
## 889  2020-04-12               South West     23
## 890  2020-04-13               South West     27
## 891  2020-04-14               South West     24
## 892  2020-04-15               South West     32
## 893  2020-04-16               South West     29
## 894  2020-04-17               South West     33
## 895  2020-04-18               South West     25
## 896  2020-04-19               South West     31
## 897  2020-04-20               South West     26
## 898  2020-04-21               South West     26
## 899  2020-04-22               South West     23
## 900  2020-04-23               South West     17
## 901  2020-04-24               South West     19
## 902  2020-04-25               South West     15
## 903  2020-04-26               South West     27
## 904  2020-04-27               South West     13
## 905  2020-04-28               South West     17
## 906  2020-04-29               South West     15
## 907  2020-04-30               South West     26
## 908  2020-05-01               South West      6
## 909  2020-05-02               South West      7
## 910  2020-05-03               South West     10
## 911  2020-05-04               South West     17
## 912  2020-05-05               South West     14
## 913  2020-05-06               South West     19
## 914  2020-05-07               South West     16
## 915  2020-05-08               South West      6
## 916  2020-05-09               South West     11
## 917  2020-05-10               South West      5
## 918  2020-05-11               South West      8
## 919  2020-05-12               South West      7
## 920  2020-05-13               South West      7
## 921  2020-05-14               South West      6
## 922  2020-05-15               South West      4
## 923  2020-05-16               South West      4
## 924  2020-05-17               South West      6
## 925  2020-05-18               South West      4
## 926  2020-05-19               South West      6
## 927  2020-05-20               South West      1
## 928  2020-05-21               South West      9
## 929  2020-05-22               South West      6
## 930  2020-05-23               South West      6
## 931  2020-05-24               South West      3
## 932  2020-05-25               South West      8
## 933  2020-05-26               South West     11
## 934  2020-05-27               South West      5
## 935  2020-05-28               South West     10
## 936  2020-05-29               South West      7
## 937  2020-05-30               South West      3
## 938  2020-05-31               South West      2
## 939  2020-06-01               South West      7
## 940  2020-06-02               South West      2
## 941  2020-06-03               South West      7
## 942  2020-06-04               South West      2
## 943  2020-06-05               South West      2
## 944  2020-06-06               South West      1
## 945  2020-06-07               South West      3
## 946  2020-06-08               South West      3
## 947  2020-06-09               South West      0
## 948  2020-06-10               South West      1
## 949  2020-06-11               South West      2
## 950  2020-06-12               South West      2
## 951  2020-06-13               South West      2
## 952  2020-06-14               South West      0
## 953  2020-06-15               South West      2
## 954  2020-06-16               South West      2
## 955  2020-06-17               South West      0
## 956  2020-06-18               South West      0
## 957  2020-06-19               South West      0
## 958  2020-06-20               South West      2
## 959  2020-06-21               South West      0
## 960  2020-06-22               South West      1
## 961  2020-06-23               South West      1
## 962  2020-06-24               South West      1
## 963  2020-06-25               South West      0
## 964  2020-06-26               South West      3
## 965  2020-06-27               South West      0
## 966  2020-06-28               South West      0
## 967  2020-06-29               South West      1
## 968  2020-06-30               South West      0
## 969  2020-07-01               South West      0
## 970  2020-07-02               South West      0
## 971  2020-07-03               South West      0
## 972  2020-07-04               South West      0
## 973  2020-07-05               South West      1
## 974  2020-07-06               South West      0
## 975  2020-07-07               South West      0
## 976  2020-07-08               South West      2
## 977  2020-07-09               South West      0
## 978  2020-07-10               South West      1
## 979  2020-07-11               South West      0
## 980  2020-07-12               South West      0
## 981  2020-07-13               South West      1
## 982  2020-07-14               South West      0
## 983  2020-07-15               South West      0
## 984  2020-07-16               South West      0
## 985  2020-07-17               South West      1
## 986  2020-07-18               South West      0
## 987  2020-07-19               South West      0

1.5 Completion date

We extract the completion date from the NHS Pathways file timestamp:


database_date <- attr(x, "timestamp")
database_date
## [1] "2020-07-20"

The completion date of the NHS Pathways data is Monday 20 Jul 2020.

1.6 Auxiliary functions

These are functions which will be used further in the analyses.

Function to estimate the generalised R-squared as the proportion of deviance explained by a given model:


## Function to calculate R2 for Poisson model
## not adjusted for model complexity but all models have the same DF here

Rsq <- function(x) {
  1 - (x$deviance / x$null.deviance)
}

Function to extract growth rates per region as well as halving times, and the associated 95% confidence intervals:


## function to extract the coefficients, find the level of the intercept,
## reconstruct the values of r, get confidence intervals

get_r <- function(model) {
  ##  extract coefficients and conf int
  out <- data.frame(r = coef(model))  %>%
    rownames_to_column("var") %>% 
    cbind(confint(model)) %>%
    filter(!grepl("day_of_week", var)) %>% 
    filter(grepl("day", var)) %>%
    rename(lower_95 = "2.5 %",
           upper_95 = "97.5 %") %>%
    mutate(var = sub("day:", "", var))
  
  ## reconstruct values: intercept + region-coefficient
  for (i in 2:nrow(out)) {
    out[i, -1] <- out[1, -1] + out[i, -1]
  }
  
  ## find the name of the intercept, restore regions names
  out <- out %>%
    mutate(nhs_region = model$xlevels$nhs_region) %>%
    select(nhs_region, everything(), -var)
  
  ## find halving times
  halving <- log(0.5) / out[,-1] %>%
    rename(halving_t = r,
           halving_t_lower_95 = lower_95,
           halving_t_upper_95 = upper_95)
  
  ## set halving times with exclusion intervals to NA
  no_halving <- out$lower_95 < 0 & out$upper_95 > 0
  halving[no_halving, ] <- NA_real_
  
  ## return all data
  cbind(out, halving)
  
}

Functions used in the correlation analysis between NHS Pathways reports and deaths:

## Function to calculate Pearson's correlation between deaths and lagged
## reports. Note that `pearson` can be replaced with `spearman` for rank
## correlation.

getcor <- function(x, ndx) {
  return(cor(x$deaths[ndx],
             x$note_lag[ndx],
             use = "complete.obs",
             method = "pearson"))
}

## Catch if sample size throws an error
getcor2 <- possibly(getcor, otherwise = NA)

getboot <- function(x) {
  result <- boot::boot.ci(boot::boot(x, getcor2, R = 1000), 
                           type = "bca")
  return(data.frame(n = sum(!is.na(x$note_lag) & !is.na(x$deaths)),
                    r = result$t0,
                    r_low = result$bca[4],
                    r_hi = result$bca[5]))
}

Function to classify the day of the week into weekend, Monday, and the rest:


## Fn to add day of week
day_of_week <- function(df) {
  df %>% 
    dplyr::mutate(day_of_week = lubridate::wday(date, label = TRUE)) %>% 
    dplyr::mutate(day_of_week = dplyr::case_when(
      day_of_week %in% c("Sat", "Sun") ~ "weekend",
      day_of_week %in% c("Mon") ~ "monday",
      !(day_of_week %in% c("Sat", "Sun", "Mon")) ~ "rest_of_week"
    ) %>% 
      factor(levels = c("rest_of_week", "monday", "weekend")))
}

Custom color palettes, color scales, and vectors of colors:


pal <- c("#006212",
         "#ae3cab",
         "#00db90",
         "#960c00",
         "#55aaff",
         "#ff7e78",
         "#00388d")

age.pal <- viridis::viridis(3,begin = 0.1, end = 0.7)

3 Comparison with deaths time series

3.1 Outline

We want to explore the correlation between NHS Pathways reports and deaths, and assess the potential for reports to be used as an early warning system for disease resurgence.

Death data are publically available. We truncate the time series to avoid bias from reporting delay - we assume a conservative delay of three weeks.

3.2 Lagged correlation

We calculate Pearson’s correlation coefficient between deaths and NHS Pathways notifications using different lags. Confidence intervals are obtained using bootstrap. Note that results were also confirmed using Spearman’s rank correlation.

First we join the NHS Pathways and death data, and aggregate over all England:

## truncate death data for reporting delay
trunc_date <- max(dth$date_report) - delay_max

dth_trunc <- dth %>%
  rename(date = date_report) %>%
  filter(date <= trunc_date) 

## join with notification data
all_data <- x %>% 
  filter(!is.na(nhs_region)) %>%
  group_by(date, nhs_region) %>%
  summarise(count = sum(count, na.rm = T)) %>%
  ungroup %>%
  inner_join(dth_trunc,
             by = c("date","nhs_region"))

all_tot <- all_data %>%
  group_by(date) %>%
  summarise(count = sum(count, na.rm = TRUE),
            deaths = sum(deaths, na.rm = TRUE)) 

We calculate correlation with lagged NHS Pathways reports from 0 to 30 days behind deaths:


## Calculate all correlations + bootstrap CIs
lag_cor <- data.frame()
for (i in 0:30) {
  
  ## lag reports
  summary <- all_tot %>% 
    mutate(note_lag = lag(count, i)) %>%
    ## calculate rank correlation and bootstrap CI
    getboot(.) %>%
    mutate(lag = i)

  lag_cor <- bind_rows(lag_cor, summary)
}

cor_vs_lag <- ggplot(lag_cor, aes(lag, r)) +
  theme_bw() +
  geom_ribbon(aes(ymin = r_low, ymax = r_hi), alpha = 0.2) +
  geom_hline(yintercept = 0, lty = "longdash") +
  geom_point() +
  geom_line() +
  labs(x = "Lag between NHS pathways and death data (days)",
       y = "Pearson's correlation") +
  large_txt
cor_vs_lag


l_opt <- which.max(lag_cor$r)

This analysis suggests that the best lag is 23 days. We then compare and plot the number of deaths reported against the number of NHS Pathways reports lagged by 23 days.


all_tot <- all_tot %>%
  rename(date_death = date) %>%
  mutate(note_lag = lag(count, lag_cor$lag[l_opt]),
         note_lag_c = (note_lag - mean(note_lag, na.rm = T)),
         date_note = lag(date_death,16))

lag_mod <- glm(deaths ~ note_lag, data = all_tot, family = "quasipoisson")

summary(lag_mod)
## 
## Call:
## glm(formula = deaths ~ note_lag, family = "quasipoisson", data = all_tot)
## 
## Deviance Residuals: 
##      Min        1Q    Median        3Q       Max  
## -13.7184   -4.8961   -0.4672    3.8101    8.6431  
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 4.557e+00  6.582e-02   69.24   <2e-16 ***
## note_lag    1.485e-05  6.862e-07   21.64   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for quasipoisson family taken to be 25.61314)
## 
##     Null deviance: 13033.9  on 79  degrees of freedom
## Residual deviance:  2114.1  on 78  degrees of freedom
##   (23 observations deleted due to missingness)
## AIC: NA
## 
## Number of Fisher Scoring iterations: 4

exp(coefficients(lag_mod))
## (Intercept)    note_lag 
##   95.336763    1.000015
exp(confint(lag_mod))
##                 2.5 %     97.5 %
## (Intercept) 83.621875 108.244969
## note_lag     1.000014   1.000016

Rsq(lag_mod)
## [1] 0.8378011

mod_fit <- as.data.frame(predict(lag_mod, type = "link", se.fit = TRUE)[1:2])

all_tot_pred <- 
  all_tot %>%
  filter(!is.na(note_lag)) %>%
  mutate(pred = mod_fit$fit,
         pred.se = mod_fit$se.fit,
         low = exp(pred - 1.96*pred.se),
         hi = exp(pred + 1.96*pred.se))


glm_fit <- all_tot_pred %>% 
    filter(!is.na(note_lag)) %>%
  ggplot(aes(x = note_lag, y = deaths)) +
  geom_point() + 
  geom_line(aes(y = exp(pred))) + 
  geom_ribbon(aes(ymin = low, ymax = hi), alpha = 0.3, col = "grey") +
  theme_bw() +
  labs(y = "Daily number of\ndeaths reported",
       x = "Daily number of NHS Pathways reports") +
  large_txt

glm_fit

4 Supplementary figures

4.1 Serial interval distribution

This is a comparison of gamma versus lognormal distribution for the serial interval used to convert r to R in our analysis. Both distributions are parameterised with mean 4.7 and standard deviation 2.9.

SI_param <- epitrix::gamma_mucv2shapescale(4.7, 2.9/4.7)
SI_distribution <- distcrete::distcrete("gamma", interval = 1,
                                        shape = SI_param$shape,
                                        scale = SI_param$scale, w = 0.5)

SI_distribution2 <- distcrete::distcrete("lnorm", interval = 1,
                                        meanlog = log(4.7),
                                        sdlog = log(2.9), w = 0.5)

SI_dist1 <- data.frame(x = SI_distribution$r(1e5)) 
SI_dist1 <- count(SI_dist1, x) %>%
    ggplot() +
    geom_col(aes(x = x, y = n)) +
    labs(x = "Serial interval (days)", y = "Frequency") +
    scale_x_continuous(breaks = seq(0, 30, 5)) +
    theme_bw()

SI_dist2 <- data.frame(x = SI_distribution2$r(1e5)) 
SI_dist2 <- count(SI_dist2, x) %>%
    ggplot() +
    geom_col(aes(x = x, y = n)) +
    labs(x = "Serial interval (days)", y = "Frequency") +
    scale_x_continuous(breaks = seq(0, 200, 20), limits = c(0, 200)) +
    theme_bw()


ggpubr::ggarrange(SI_dist1,
                  SI_dist2,
                  nrow = 1,
                  labels = "AUTO") 

4.2 Sensitivity analysis - 7 or 21 days moving window

We reproduce the window analysis with either a 7 or 21 days window for sensitivity purposes.

First with the 7 days window:

## set moving time window (1/2/3 weeks)
w <- 7

# create empty df
r_all_sliding_7days <- NULL

## make data for model
x_model_all_moving <- x %>%
  filter(!is.na(nhs_region)) %>% 
  group_by(date, nhs_region) %>%
  summarise(n = sum(count)) 

unique_dates <- unique(x_model_all_moving$date)

for (i in 1:(length(unique_dates) - w)) {
  
  date_i <- unique_dates[i]
  
  date_i_max <- date_i + w
  
  model_data <- x_model_all_moving %>%
    filter(date >= date_i & date < date_i_max) %>%
    mutate(day = as.integer(date - date_i)) %>% 
    day_of_week()
  
  
  mod <- glm(n ~ day * nhs_region + day_of_week,
             data = model_data,
             family = 'quasipoisson')
  
  # get growth rate
  r <- get_r(mod)
  r$w_min <- date_i
  r$w_max <- date_i_max
  
  # combine all estimates
  r_all_sliding_7days <- bind_rows(r_all_sliding_7days, r)
  
}

#serial interval distribution
SI_param = epitrix::gamma_mucv2shapescale(4.7, 2.9/4.7)
SI_distribution <- distcrete::distcrete("gamma", interval = 1,
                                        shape = SI_param$shape,
                                        scale = SI_param$scale,
                                        w = 0.5)

#convert growth rates r to R0
r_all_sliding_7days <- r_all_sliding_7days %>%
  mutate(R = epitrix::r2R0(r, SI_distribution),
         R_lower_95 = epitrix::r2R0(lower_95, SI_distribution),
         R_upper_95 = epitrix::r2R0(upper_95, SI_distribution))
# plot
plot_growth <-
  r_all_sliding_7days %>%
  ggplot(aes(x = w_max, y = r)) +
  geom_ribbon(aes(ymin = lower_95, ymax = upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(yintercept = 0, linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0.5,0.5, "cm")) +
  guides(colour = guide_legend(title = "",override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "",
       y = "Estimated daily growth rate (r)") +
  scale_colour_manual(values = pal)
plot_R <- r_all_sliding_7days %>%
  ggplot(aes(x = w_max, y = R)) +
  geom_ribbon(aes(ymin = R_lower_95, ymax = R_upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(yintercept = 1, linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0.5,0.5, "cm")) +
  guides(color = guide_legend(title = "", override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "",
       y = "Estimated effective reproduction\nnumber (Re)") +
  scale_colour_manual(values = pal)

R <- r_all_sliding_7days %>%
  mutate(lower_95 = R_lower_95, 
         upper_95 = R_upper_95,
         value = R,
         measure = "R",
         reference = 1)

r_R <- r_all_sliding_7days %>%
  mutate(measure = "r",
         value = r,
         reference = 0) %>%
  bind_rows(R)

r_R_7 <- r_R %>%
  ggplot(aes(x = w_max, y = value)) +
  geom_ribbon(aes(ymin = lower_95, ymax = upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(aes(yintercept = reference), linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0,0, "cm"),
        strip.background = element_blank(),
        strip.placement = "outside"
  ) +
  guides(color = guide_legend(title = "", override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "", y = "") +
  scale_colour_manual(values = pal) +
  facet_grid(rows = vars(measure),
             scales = "free_y",
             switch = "y",
             labeller = as_labeller(c(r = "Daily growth rate (r)",
                                      R = "Effective reproduction\nnumber (Re)")))

Then with the 21 days window:

## set moving time window (1/2/3 weeks)
w <- 21

# create empty df
r_all_sliding_21days <- NULL

## make data for model
x_model_all_moving <- x %>%
  filter(!is.na(nhs_region)) %>% 
  group_by(date, nhs_region) %>%
  summarise(n = sum(count)) 

unique_dates <- unique(x_model_all_moving$date)

for (i in 1:(length(unique_dates) - w)) {
  
  date_i <- unique_dates[i]
  
  date_i_max <- date_i + w
  
  model_data <- x_model_all_moving %>%
    filter(date >= date_i & date < date_i_max) %>%
    mutate(day = as.integer(date - date_i)) %>% 
    day_of_week()
  
  
  mod <- glm(n ~ day * nhs_region + day_of_week,
             data = model_data,
             family = 'quasipoisson')
  
  # get growth rate
  r <- get_r(mod)
  r$w_min <- date_i
  r$w_max <- date_i_max
  
  # combine all estimates
  r_all_sliding_21days <- bind_rows(r_all_sliding_21days, r)
  
}

#serial interval distribution
SI_param = epitrix::gamma_mucv2shapescale(4.7, 2.9/4.7)
SI_distribution <- distcrete::distcrete("gamma", interval = 1,
                                        shape = SI_param$shape,
                                        scale = SI_param$scale,
                                        w = 0.5)

#convert growth rates r to R0
r_all_sliding_21days <- r_all_sliding_21days %>%
  mutate(R = epitrix::r2R0(r, SI_distribution),
         R_lower_95 = epitrix::r2R0(lower_95, SI_distribution),
         R_upper_95 = epitrix::r2R0(upper_95, SI_distribution))
# plot
plot_growth <-
  r_all_sliding_21days %>%
  ggplot(aes(x = w_max, y = r)) +
  geom_ribbon(aes(ymin = lower_95, ymax = upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(yintercept = 0, linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0.5,0.5, "cm")) +
  guides(colour = guide_legend(title = "",override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "",
       y = "Estimated daily growth rate (r)") +
  scale_colour_manual(values = pal)
# plot
plot_R <-
  r_all_sliding_21days %>%
  ggplot(aes(x = w_max, y = R)) +
  geom_ribbon(aes(ymin = R_lower_95, ymax = R_upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(yintercept = 1, linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0.5,0.5, "cm")) +
  guides(color = guide_legend(title = "", override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "",
       y = "Estimated effective reproduction\nnumber (Re)") +
  scale_colour_manual(values = pal)

R <- r_all_sliding_21days %>%
  mutate(lower_95 = R_lower_95, 
         upper_95 = R_upper_95,
         value = R,
         measure = "R",
         reference = 1)

r_R <- r_all_sliding_21days %>%
  mutate(measure = "r",
         value = r,
         reference = 0) %>%
  bind_rows(R)

r_R_21 <- r_R %>%
  ggplot(aes(x = w_max, y = value)) +
  geom_ribbon(aes(ymin = lower_95, ymax = upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(aes(yintercept = reference), linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0,0, "cm"),
        strip.background = element_blank(),
        strip.placement = "outside"
  ) +
  guides(color = guide_legend(title = "", override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "", y = "") +
  scale_colour_manual(values = pal) +
  facet_grid(rows = vars(measure),
             scales = "free_y",
             switch = "y",
             labeller = as_labeller(c(r = "Daily growth rate (r)",
                                      R = "Effective reproduction\nnumber (Re)")))

And we combine both outputs into a single plot:


ggpubr::ggarrange(r_R_7,
                  r_R_21,
                  nrow = 2,
                  labels = "AUTO",
                  common.legend = TRUE,
                  legend = "bottom") 

4.3 Correlation between NHS Pathways reports and deaths by NHS region


lag_cor_reg <- data.frame()

for (i in 0:30) {

  summary <-
    all_data %>%
    group_by(nhs_region) %>%
    mutate(note_lag = lag(count, i)) %>%
    ## calculate rank correlation and bootstrap CI for each region
    group_modify(~getboot(.x)) %>%
    mutate(lag = i)
  
  lag_cor_reg <- bind_rows(lag_cor_reg, summary)
}

cor_vs_lag_reg <- 
lag_cor_reg %>%
ggplot(aes(lag, r, col = nhs_region)) +
  geom_hline(yintercept = 0, lty = "longdash") +
  geom_ribbon(aes(ymin = r_low, ymax = r_hi, col = NULL, fill = nhs_region), alpha = 0.2) +
  geom_point() +
  geom_line() +
  facet_wrap(~nhs_region) +
  scale_color_manual(values = pal) +
  scale_fill_manual(values = pal, guide = F) +  
  theme_bw() +
  labs(x = "Lag between NHS pathways and death data (days)", y = "Pearson's correlation", col = "NHS region") +
  theme(legend.position = "bottom") +
  guides(color = guide_legend(override.aes = list(fill = NA)))

cor_vs_lag_reg

5 Export data

We save the tables created during our analysis:


if (!dir.exists("excel_tables")) {
  dir.create("excel_tables")
}


## list all tables, and loop over export
tables_to_export <- c("r_all_sliding", "lag_cor")

for (e in tables_to_export) {
  rio::export(get(e),
              file.path("excel_tables",
                        paste0(e, ".xlsx")))
}

## also export result from regression on lagged data 
rio::export(lag_mod, file.path("excel_tables", "lag_mod.rds"))

6 System information

6.1 Outline

The following information documents the system on which the document was compiled.

6.2 System

This provides information on the operating system.

Sys.info()
##                                                                                            sysname 
##                                                                                           "Darwin" 
##                                                                                            release 
##                                                                                           "19.5.0" 
##                                                                                            version 
## "Darwin Kernel Version 19.5.0: Tue May 26 20:41:44 PDT 2020; root:xnu-6153.121.2~2/RELEASE_X86_64" 
##                                                                                           nodename 
##                                                                                   "Mac-1494.local" 
##                                                                                            machine 
##                                                                                           "x86_64" 
##                                                                                              login 
##                                                                                             "root" 
##                                                                                               user 
##                                                                                           "runner" 
##                                                                                     effective_user 
##                                                                                           "runner"

6.3 R environment

This provides information on the version of R used:

R.version
##                _                           
## platform       x86_64-apple-darwin17.0     
## arch           x86_64                      
## os             darwin17.0                  
## system         x86_64, darwin17.0          
## status                                     
## major          4                           
## minor          0.2                         
## year           2020                        
## month          06                          
## day            22                          
## svn rev        78730                       
## language       R                           
## version.string R version 4.0.2 (2020-06-22)
## nickname       Taking Off Again

6.4 R packages

This provides information on the packages used:

sessionInfo()
## R version 4.0.2 (2020-06-22)
## Platform: x86_64-apple-darwin17.0 (64-bit)
## Running under: macOS Catalina 10.15.5
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] ggnewscale_0.4.1     ggpubr_0.4.0         lubridate_1.7.9     
##  [4] chngpt_2020.5-21     cyphr_1.1.0          DT_0.14             
##  [7] kableExtra_1.1.0     janitor_2.0.1        remotes_2.2.0       
## [10] projections_0.5.1    earlyR_0.0.1         epitrix_0.2.2       
## [13] distcrete_1.0.3      incidence_1.7.1      rio_0.5.16          
## [16] reshape2_1.4.4       rvest_0.3.5          xml2_1.3.2          
## [19] linelist_0.0.40.9000 forcats_0.5.0        stringr_1.4.0       
## [22] dplyr_1.0.0          purrr_0.3.4          readr_1.3.1         
## [25] tidyr_1.1.0          tibble_3.0.3         ggplot2_3.3.2       
## [28] tidyverse_1.3.0      here_0.1             reportfactory_0.0.5 
## 
## loaded via a namespace (and not attached):
##  [1] nlme_3.1-148      fs_1.4.2          webshot_0.5.2     httr_1.4.2       
##  [5] rprojroot_1.3-2   tools_4.0.2       backports_1.1.8   utf8_1.1.4       
##  [9] R6_2.4.1          mgcv_1.8-31       DBI_1.1.0         colorspace_1.4-1 
## [13] withr_2.2.0       gridExtra_2.3     tidyselect_1.1.0  sodium_1.1       
## [17] curl_4.3          compiler_4.0.2    cli_2.0.2         labeling_0.3     
## [21] matchmaker_0.1.1  scales_1.1.1      digest_0.6.25     foreign_0.8-80   
## [25] rmarkdown_2.3     pkgconfig_2.0.3   htmltools_0.5.0   dbplyr_1.4.4     
## [29] htmlwidgets_1.5.1 rlang_0.4.7       readxl_1.3.1      rstudioapi_0.11  
## [33] farver_2.0.3      generics_0.0.2    jsonlite_1.7.0    crosstalk_1.1.0.1
## [37] car_3.0-8         zip_2.0.4         magrittr_1.5      kyotil_2019.11-22
## [41] Matrix_1.2-18     Rcpp_1.0.5        munsell_0.5.0     fansi_0.4.1      
## [45] viridis_0.5.1     abind_1.4-5       lifecycle_0.2.0   stringi_1.4.6    
## [49] yaml_2.2.1        carData_3.0-4     snakecase_0.11.0  MASS_7.3-51.6    
## [53] plyr_1.8.6        grid_4.0.2        blob_1.2.1        crayon_1.3.4     
## [57] lattice_0.20-41   cowplot_1.0.0     splines_4.0.2     haven_2.3.1      
## [61] hms_0.5.3         knitr_1.29        pillar_1.4.6      boot_1.3-25      
## [65] ggsignif_0.6.0    reprex_0.3.0      glue_1.4.1        evaluate_0.14    
## [69] data.table_1.12.8 modelr_0.1.8      vctrs_0.3.2       selectr_0.4-2    
## [73] cellranger_1.1.0  gtable_0.3.0      assertthat_0.2.1  xfun_0.15        
## [77] openxlsx_4.1.5    broom_0.7.0       rstatix_0.6.0     survival_3.1-12  
## [81] viridisLite_0.3.0 ellipsis_0.3.1